Papers with question answering models
Cooperative Self-training of Machine Reading Comprehension (2022.naacl-main)
Copied to clipboard
| Challenge: | Pretrained language models provide high-quality contextualized word embeddings, but training question answering models requires large amounts of annotated data for specific domains. |
| Approach: | They propose a framework for automatically generating more non-trivial question-answer pairs to improve model performance. |
| Outcome: | The proposed framework outperforms state-of-the-art (SOTA) pretrained language models and transfer learning approaches on standard question-answering benchmarks. |
Simple and Effective Multi-Paragraph Reading Comprehension (P18-1)
Copied to clipboard
| Challenge: | Existing question answering models cannot scale beyond short paragraphs, so adapting a model to document-level input is difficult. |
| Approach: | They propose a method of adapting neural paragraph-level question answering models to document input. |
| Outcome: | The proposed method achieves state-of-the-art on TriviaQA and SQuAD and a 10 point gain on SQuADA. |
Evaluating Theory of Mind in Question Answering (D18-1)
Copied to clipboard
| Challenge: | a dataset is proposed for question answering models with respect to their capacity to reason about beliefs. |
| Approach: | They propose a dataset for evaluating question answering models with respect to their capacity to reason about beliefs. |
| Outcome: | The proposed dataset is inspired by theory-of-mind experiments that examine whether children are able to reason about beliefs of others. |
Disfl-QA: A Benchmark Dataset for Understanding Disfluencies in Question Answering (2021.findings-acl)
Copied to clipboard
| Challenge: | Disfluencies are an under-studied topic in NLP, even though it is ubiquitous in human conversation. |
| Approach: | They propose a challenge question answering dataset where humans introduce contextual disfluencies in previously fluent questions. |
| Outcome: | The proposed dataset shows that existing models degrade significantly when tested on DISFL-QA in a zero-shot setting. |
Neural Natural Logic Inference for Interpretable Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing question answering models are based on textual entailment tasks . prior work has focused on QA on premise-based questions . |
| Approach: | They propose a neural-symbolic QA approach that integrates natural logic reasoning within deep learning architectures towards developing effective question answering models. |
| Outcome: | The proposed model outperforms previous work on multiple-choice science questions . it integrates natural logic reasoning within deep learning architectures to build proof paths . |
WorldTree: A Corpus of Explanation Graphs for Elementary Science Questions supporting Multi-hop Inference (L18-1)
Copied to clipboard
| Challenge: | Existing methods of automated inference do not provide enough gold explanations to train models . standardized science exams are a challenge task for question answering . |
| Approach: | They propose to manually construct a corpus of explanations for standardized science exams . they also provide an explanation-centered tablestore that contains the knowledge to construct these explanations . |
| Outcome: | The proposed model provides detailed explanations for standardized science exams . the authors show that the proposed model can be trained on the basis of gold explanations . |
Exploring The Landscape of Distributional Robustness for Question Answering Models (2022.findings-emnlp)
Copied to clipboard
Anas Awadalla, Mitchell Wortsman, Gabriel Ilharco, Sewon Min, Ian Magnusson, Hannaneh Hajishirzi, Ludwig Schmidt
| Challenge: | Existing methods for predicting distributional robustness fail to generalize reliably in a variety of test conditions. |
| Approach: | They conduct a large empirical evaluation to investigate the landscape of distributional robustness in question answering. |
| Outcome: | The proposed methods are more robust to distribution shifts than fully fine-tuned models, and few-shot prompt models exhibit better robustness than few- shot prompt models. |
QuestEval: Summarization Asks for Fact-based Evaluation (2021.emnlp-main)
Copied to clipboard
Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano, Alex Wang, Patrick Gallinari
| Challenge: | Existing evaluation metrics for summarization evaluation are limited and do not correlate well with human judgments. |
| Approach: | They propose to extend existing evaluation metrics to include question answering models to assess whether a summary contains all relevant information in its source document. |
| Outcome: | The proposed framework significantly improves the correlation with human judgments over four evaluation dimensions. |
Cross-Policy Compliance Detection via Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Policy compliance detection is the task of ensuring that a scenario conforms to a policy. |
| Approach: | They propose to decompose policy compliance detection into question answering . they propose to use an existing dataset to augment expert annotations . |
| Outcome: | The proposed approach improves accuracy in cross-policy setups, especially when policies are unseen in training. |
Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a new approach to generate adversarial data is needed to improve question answering models . crowdworkers can fool a model only 8.8% of the time, compared to 17.6% for a trained model without synthetic data. |
| Approach: | They develop a pipeline that generates questions and then filters or labels them to improve quality. |
| Outcome: | The proposed approach improves state-of-the-art on a human-written adversarial dataset by 3.7F1 and improves model generalisation on nine of the twelve MRQA datasets. |
Multi-Fact Correction in Abstractive Text Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing abstractive summarization systems generate incorrect facts with respect to the source text. |
| Approach: | They propose a suite of two factual correction models that leverages question-answering knowledge to make corrections in system-generated summaries via span selection. |
| Outcome: | The proposed model improves factuality of news summarization without sacrificing summary quality. |